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Designing multimedia learning systems for adult learners: 
Basic skills with a workforce emphasis 



John P. Sabatini 

National Center on Adult Literacy 
University of Pennsylvania 



Abstract 

This working paper reviews and analyzes results of a formative evaluation of the 
LiteracyLink Workplace Essential Skills (WES) learning system conducted in the fall of 
1998. In large, complex technology projects, even formative evaluation results require 
interpretation and judgment regarding how they should be applied in revisions to the 
design. This paper provides a glimpse into that part of the research and development 
process rarely reported in technical reports or journal articles published after a project has 
been completed. Five design issues are discussed and interpreted: (a) redundancy versus 
reinforcement across media; (b) readability; (c) responses to the video, workbook, and 
online materials as stand alone products; (d) pedagogical assumptions underlying the 
writing and media design; and (e) effectiveness of the teacher’s guide. Issues are analyzed 
in relation to design principles underlying development, and how and whether revisions 
should be applied to the WES products. 



National Center on Adult Literacy/Working Paper Series 
WP00-01 



page 3 



Designing multimedia learning systems 



March 2001 



Introduction 

In theory, the design of technological learning systems is based on principles and 
guidelines derived from theory, empirical research, experience, and the constraints or 
possibilities of the technologies used. Throughout the process, specific design decisions 
are made based on the designer’s best judgment of how to apply principles and guidelines 
in practice - a task that requires drawing inferences and generalizations. In practice, 
innovations and assumptions seem to outnumber empirically proven guidelines, especially 
when a new technology is just emerging or is being introduced to a new audience. 
Formative evaluation provides the mechanism for testing the quality of those design 
innovations and assumptions. However, in large, complex technology systems and 
projects, formative evaluation results require interpretation and judgment regarding how 
revisions to the product design should be implemented. 

The purpose of this working paper is to review lessons learned about the design of 
the LiteracyLink Workplace Essential Skills (WES) learning system, as a result of a 
formative evaluation conducted by the Institute for Social Research (Johnston & Young, 
1999) in the fall of 1998. The paper provides a glimpse into the process of research and 
development rarely reported in the technical reports or research journal articles that are 
published after a project has been completed. The issues raised will be of specific interest to 
members of the adult literacy community - researchers, designers, and practitioners - as 
well as instructional designers and evaluators who are involved in technology projects. 

Background 



The LiteracyLink project 

The U.S. Department of Education, through its Star Schools grants program, has 
funded the LiteracyLink project, a joint venture of PBS, Kentucky Educational Television 
(KET), the National Center on Adult Literacy (NCAL) at the University of Pennsylvania, 
and the Kentucky Department of Education. The project uses video, online, and Internet 
computer technology to help adults receive literacy instruction and gain a high school 
diploma or equivalent. 

One component of the LiteracyLink project is the Workplace Essential Skills (WES) 
multimedia learning system. Multimedia refers to the integration of media such as text, 
sound, graphics, animation, video, and images in a computer system (Jonassen & Reeves, 
1996). The WES multimedia system includes video, workbook/text, and a website. The 
video series consists of 26 programs addressing basic skills in reading, writing, 
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communication, and mathematics in a workplace context. The series also addresses issues 
of job searching, career planning, and workplace orientation. Four printed workbooks 
serve as companion pieces. The online portion is designed to deliver instructional content 
integrating clips from the video and other Internet resources in easy-to-use electronic 
formats. The learning activities are designed to encourage interactive access to a wide range 
of relevant materials both in the LiteracyLink website and on the Internet at large. The WES 
system was launched nationally in September 1999. 

Applying adult learning theory to the design of technological learning systems 

Stites (1998) reviewed the literature on adult learning and development, and derived 
four general principles relevant to learning and instruction with technology. He 
recommends that designers (a) plan for learning across the lifespan, (b) identify distinctive 
characteristics of adult learners, (c) emphasize the goals of adults, and (d) create adult- 
situated contexts for learning. Stites draws on such notable theorists as Freire (1970), 
Houle (1984), Knowles (1990), and Mezirow (1995), as well as cognitive learning 
theories (e.g., Collins, Brown, & Newman, 1989) to support his synthesis. 

In considering how to translate these principles into an instructional design 
delivered via a multimedia system (in this case, the media consisting of print, video, and 
web-based learning materials), foremost must be considerations of the strengths each 
medium affords. Video is effective for setting context, modeling, motivating, and 
illustrating concepts and examples. Texts and workbooks are the traditional workhorses of 
print pedagogy. They organize and sequence content and present information, 
explanations, examples, questions, activities, and assessments in a durable, portable form. 
Strengths of computer media include the capabilities of data processing, storage, and 
retrieval, as well as generative applications such as word processing, graphic design, 
spreadsheets, databases, and other programming. Computers can store and display print, 
graphics, photos, animation, and video. Networked computers provide the added 
capabilities of communication and access to wider informational resources. Internet- 
connected computers provide access to vast amounts of information and resources that are 
otherwise unattainable in a single instructional setting. 

In summary, multimedia products provide a variety of mechanisms for creating 
practical, flexible implementation of adult learning principles to support multiple learning 
environments. The role of the designer is to select and apply a set of features afforded by 
each medium into a design that facilitates effective learning. 
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Design features in the WES system 

The LiteracyLink/WES system is a hybrid of classroom-based activities and 
individual tutorial instruction. The WES videos use a combination of documentary footage 
of real workplaces including interviews with workers, supervisors, and experts that create 
authentic settings, models, and motivation for conceptual content. Dramatized scenarios 
were also used to show problem situations and model how adults might go about solving 
them by applying knowledge and skills. Teachers were provided with lesson plans that 
engage groups of learners in Before, During, and After discussions and activities related to 
the video programs. 

Print publications for the series apply a traditional instructional workbook text 
design by presenting basic concepts and vocabularies, describing authentic workplace case 
scenarios, and providing open-ended and closed-end activities to help learners practice 
applying skills. Each workbook chapter began with an introduction and overview of the 
corresponding video program, as well as statements of unit objectives and discussion 
questions. Pre- and post-tests in the workbook helped learners self-assess their 
understanding and progress in learning basic concepts. 

At the time of the formative evaluation, the online system was primarily an 
individual tutorial instruction system. Learners worked one at a time on online activities and 
saved their results to a personal electronic file. At the time, there were no tools for learners 
to share and communicate with others online. In the absence of online communication 
tools, group-oriented social interactive activities were based on suggested activities in the 
online instruction and teacher’s guide that facilitated social learning and communication. 

The opportunity to visit websites external to the LiteracyLink site, and to potentially 
communicate with others through those sites, was a feature of the online design that 
introduced learners to the wider world of resources available on the Internet, as well as 
created an awareness of the different navigational and organizational structures and 
communication possibilities of websites. 

In sum, the LiteracyLink/WES learning environment was designed to use multiple 
media to maximize opportunities for learner-centered, social, contextualized, and problem- 
focused learning. Before attempting to validate that applying these principles in WES 
would lead to instructional gains (i.e., summative evaluation), even more basic issues of 
usability, appeal, and comprehensibility of design needed to be addressed (Flagg, 1990). 
For this reason, a formative evaluation study piloting a subset of the WES products was 
conducted. 
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Purpose of formative evaluation and research questions 

The LiteracyLink Star Schools project was funded to produce a variety of 
multimedia materials (video, print, and Internet-based) directed at adult literacy learners and 
teachers. While video and print media are familiar classroom curriculum materials for this 
audience, computers and the Internet are still novel classroom tools for most. In adult 
education, computers are more often used for individual tutorials, rather than integrated into 
the day-to-day classroom environment (Ginsburg, 1998). There is little adult literacy 
research on how computers are to be integrated in classroom-wide instruction, though there 
are ample examples and guides from the K-12 literature. 

The broad research questions guiding the formative evaluation of LiteracyLink/WES 
(Johnston & Young, 1999) were as follows: 

• Video - Is the video interesting and comprehensible? Does it stimulate discussion? 
Do reactions to the videos remain constant for the duration of the course? 

• Workbook - Is the workbook interesting and comprehensible? Are the 
tasks/exercises easily understood? Are die tasks valued? Do reactions to the videos 
remain constant for the duration of the course? 

• Online Materials - Can learners access and navigate the online materials easily? Are 
the tasks easily understood and completed? Do learners value the online activities? 
Do reactions to the videos remain constant for the duration of the course? 

Emerging from these broad questions were more specific issues of the appeal, usability, 
and comprehensibility of the design and content. After describing the design and methods 
used to conduct the formative evaluation, we will return to a discussion of the specific 
issues that emerged. 



Methods 



Curriculum and materials piloted 

The pilot consisted of two 4-6 week, mini-courses based on available products in 
fall of 1998. These courses were Finding a Job (Units 1-5) and Workplace Writing (Units 
13-15). Each unit consisted of one half-hour video, one workbook chapter, and one set of 
online activities (typically 2 to 3 individual lesson activities). Teachers attempted to cover 
one unit a week in the Finding a Job mini-course, and about one unit every week and a half 
for Workplace Writing. In addition, each teacher received one day of training by 
LiteracyLink staff as well as a teacher’s guide. The teacher’s guide consisted of an 
overview of the project, introductory sections describing the navigation and structure of the 
Internet site, as well as brief Before, During, and After lesson plan activity selections for 
each video, workbook chapter subsection, and online video activity. 
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Data source 

LiteracyLink is working with 25 innovation sites clustered in the vicinity of Los 
Angeles, CA; Buffalo, NY; Des Moines, IA; Lexington, KY; Philadelphia, PA; and Reno, 
NV. The innovation sites consist of local adult literacy programs sponsored by community- 
based organizations, community colleges, or private or public schools. The pilot test 
sample consisted of 14 classrooms (7 each teaching Finding a Job and Workplace Writing) 
and 19 teachers (3 classes were team-taught) who volunteered and met selection criteria for 
this evaluation. The criteria consisted of being able to recruit 5-15 learners, pilot materials 
for 4-6 weeks, attend a one-day training session, and provide weekly feedback. Some 
teachers had been participating in the LiteracyLink project for some time and were familiar 
with goals, approaches, and existing products; others were introduced to the project as part 
of this formative pilot. 

The total number of learners as of week one of the mini-courses was 90, with 66 
remaining at the end, for an attrition rate of 27%. This attrition rate is consistent with other 
estimates of adult literacy programs. The learner sample was 47% male, with 48% between 
16-18 years of age, 26% between 19-35, and 24% 36 and older. The highest grade level 
was between 9-1 1 for 63%, with 8% less than 9, and 29% 12th grade or more based on 
placement testing or teacher estimations. 

Data collection and implementation procedures 

Data was collected from both teachers and learners. Data collection from learners 
consisted of a baseline questionnaire; weekly diary of time spent, readability of materials 
and clarity of tasks, and perceived learning; and an end-of-course survey including 
perceived learning and course rating. Data collection from teachers consisted of weekly e- 
mail questionnaires of teaching/leaming issues and reflections on the teacher’s guide, and 
an end-of-course survey on teaching/leaming issues and recommendations for adoption. 
Classroom observations were also conducted. All student online activities were collected 
and saved as part of individual student records. 

Classroom teachers selected for the pilot received a one-day training by 
LiteracyLink staff. The training was designed to be hands-on demonstrations of effective, 
integrated classroom uses of the video, workbook, and online system. Teachers also 
received a visit from LiteracyLink technical staff to help configure classroom technologies. 
Teachers had access to a technical support hotline at PBS for answering both technology 
and instructional questions. 
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Results 

Johnston and Young (1999) raised a number of issues that might need to be 
addressed or adjusted to increase the usefulness of the WES products. These are the 
following: 

• redundancy versus reinforcement across the three media 

• readability of the products 

• responses to the video, workbook, and online materials as stand alone products 

• pedagogical assumptions underlying the writing and media design 

• effectiveness of the teacher’s guide 

In the following sections, the design principle applied to the products is presented, 
followed by summary of the general results reported by Johnston and Young, and then a 
discussion of implications and interpretations of the design team. 

Redundancy versus reinforcement across the three media 

There was a high level of redundancy built into the WES design as a means of 
building familiarity, as a reinforcement of content, and to maximize the potential for each 
media to be used as a stand alone product. One reason for the high level of redundancy was 
to allow for more flexible mix and match uses of the different media products. We 
anticipated that the LiteracyLinkAVES materials might be configured differently based on 
the availability or portability of media, the nature of the program or classroom organization, 
and/or individual differences in learner needs or preferences. For example, regarding 
access to media and program organization, learners may view videos at home, do 
workbook activities as homework, and work online in class. Alternatively, learners may 
watch and discuss the video in class, work in small groups on workbook activities, then do 
online activities in a library or tech lab. 

Regarding individual differences, learners who have a stronger need for in-depth 
work on a topic such as interviewing may benefit from attempting multiple online and 
workbook activities, each of which introduces the same concepts, but requires different 
practice and performance tasks to reinforce and expand the depth of understanding. On the 
other hand, many learners may master the level of conceptual materials and “essential 
skills” after working through just one form of the materials. 

Johnston and Young (1999) reported that teachers felt that materials were repetitive 
past the point of simply reinforcing and that there was too much material to cover in each 
unit. The pilot teachers were asked and trained to present the curriculum in a highly 
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structured, mostly linear plan based on completing each unit in sequence by viewing the 
video, then mixing workbook and online activities. The high level of redundancy was most 
apparent when the curriculum was implemented in this linear, comprehensive classroom 
course approach. Although our preparation of teachers encouraged flexible configurations, 
we left teachers with little time to familiarize and plan for a more flexible approach, and no 
doubt left the impression that coverage of all the materials was important or necessary for 
effective learning and for evaluation purposes. 

Two sources of the redundancy problem were addressed in a subsequent redesign 
of the online site. First, each instructional unit online began with a lesson plan page that 
described and linked not only to the online learning activities, but also to anchor, video, 
and workbook review activities. These three additional “activities” were informational 
supports for using the video and workbooks - activities that the teacher may have 
introduced and conducted before the learner went online. These were stripped in later 
versions, simplifying the look and feel of the online material, as well as reducing the 
redundant information. 

The second source of redundancy was the result of navigational and graphic design. 
The headers at the top of each page did not clearly present information to help the learners 
orient themselves and locate what part of the curriculum (unit or activity) they were doing. 
Every webpage looked almost exactly the same. In addition, there were graphic text boxes 
at the top of each page that reviewed the correspondence between video program, 
workbook pages, and online objectives. The graphic boxes themselves were large, taking 
up most of the screen size, especially on the small monitors that many of the learners were 
using. All subsequent designs provided much cleaner and clearer navigational and location 
markers. 

Changes to the product also included more targeted teacher support in the forms of 
professional development, implementation guides, assessment and management tools, and 
technical support to foster teacher decision making about how best to implement 
LiteracyLink/WES to support specific instructional goals. Assessments to help a teacher 
and learner judge mastery of specific concepts and objectives are also under development to 
help facilitate selective use of materials rather than constraining them to use a 
comprehensive, linear progression through the materials. 

Readability of the products 

All designers of educational materials for adults are concerned about the difficulty, 
challenge, and complexity level of materials and texts. One metric of difficulty is 
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readability. Traditional readability formulas are based on word, sentence, and paragraph 
length. Such formulas are correlated with text difficulty because more difficult texts tend to 
have longer words, sentences, and paragraphs. However, readability formulas are not 
totally reliable estimations of text difficulty. Furthermore, attempts to simplify a complex 
text by shortening words and sentences, tends to create less coherent texts, impeding rather 
than improving comprehensibility (Beck & McKeown, 1986). 

The authors of the various LiteracyLink products made an effort to use simple, 
concise language and to provide glossary and other supports whenever complex terms or 
concepts were introduced. Readability formulas applied to workbook and online text 
generally rated materials as between 6 th and 8 th grade reading levels. 

Johnston and Young (1999) reported that teachers and learners found that 
workbook and online materials required reading skills above the targeted grade level. 
Learners self-reported experiencing difficulty reading text at an average rate of 23% for the 
workbooks, 24% for online video activities (that is activities based on the video program), 
and 31% for Internet activities (that is, activities based on an Internet site external to 
LiteracyLink, in which reading level could not be controlled). Johnston and Young 
recommend an extra step in the design process whereby text is critiqued by reading experts. 

The recommendation of Johnston and Young (1999) is sensible as is the need to re- 
examine the level of materials. In defense of the authors of adult educational materials, 
writing readable texts for adults is challenging. Whereas texts for young children are 
populated by bunnies, wagons, and other familiar objects and events easily described in 
plain language, adults, regardless of their personal reading levels, live in a world that uses 
complex and specific terms and concepts in the spheres of work and community. Only so 
much simplification can be achieved when the goal is to introduce authentic and relevant 
adult world information. We take encouragement that 69 to 76% of learners judged the 
materials just right. We have no way to assess what percentage of this group may have 
found the materials too easy, nor the impact of decreasing difficulty on their interest and 
motivation. 

Johnston and Young’s (1999) observations made us reflect more about the issue of 
difficulty and challenge level in adult learning in general, beyond the notion of text 
readability. First, embedded in text difficulty is the issue of complexity of the writing 
versus complexity of concepts introduced. In general, of the four online sites rated most 
difficult by learners, three had a Flesch readability index of 5. 1 and one had an index of 
6.4. This more difficult activity to read (6.4) was about “Paperwork” and had as one of its 
objectives to “practice getting the information you need to fill out difficult forms” and 
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introduced the terms and concepts of document, memo, form, reservation form, billing 
record, inventory, and itinerary. 

In this case, any design change may trade-off perceived text difficulty and real 
world authenticity, since the real world texts and tasks in this skill area have equal or 
greater complexity and difficulty that the instructional activity presented. We still hold to the 
principle that introducing learners to external websites is of the highest priority and value to 
their learning because it orients them to authentic texts and tasks, even though we 
acknowledge that we cannot as easily control the readability and difficulty level of such 
external sites. 

Interestingly, this activity was rated as Difficult by 23% as compared to 41% who 
rated as Difficult another activity in the same unit entitled “Writing a First Draff,” which has 
a 5. 1 index. This seems to be evidence that the difficulty being judged by learners is as 
much about task as it is text. 

Johnston and Young’s report also led us to reflect on the mixed profiles of 
individual readers and the mixed reading abilities that teachers often find within a single 
classroom. Johnston and Young (1999) used teachers’ judgments of learners’ reading 
levels because external assessments were not available. No doubt for most classroom 
decision making the seasoned judgment of an experienced teacher is a valid guide to 
choosing curricular and instructional materials within their group of students. However, 
there is reason to doubt that teachers are able to provide normative judgments of their 
learners’ reading levels for cross-class comparisons such as this. This is evident in the table 
reproduced below by the fact that the students that teachers judged to be at a 7 th to 8 th grade 
reading level found texts more difficult than the students teachers judged to be at 5 th or 6 th 
grade. Either the 7 th -8 th grade readers perceived texts as more difficult than the 5 th -6 th grade 
readers or the teachers’ normative judgements of reading ability were not highly reliable 
across these two groups. 

Percentage of Students Experiencing Difficulty Reading the Text bv Teacher Judgment of 
Students Reading Level (Johnston & Young, 1999) 

Teacher Judgment of Student Reading Level 





< 5th Grade 


5th-6th 


7th-8th 


>8th 


Total 


Workbook 


40% 


25% 


36% 


15% 


23% 


Online Video 


32% 


26% 


27% 


20% 


24% 


Internet 


38% 


27% 


35% 


30% 


31% 
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Regarding mixed ability levels, consider the Johnston and Young (1999) 

description of the how teachers assisted some learners. They report: 

At other times, teachers sat with students helping them track print and sound out 
words. Tracking print involves using a finger or writing implement to point to 
individual words as a means of maintaining place in text and as a means of helping 
novice readers parse words into syllables to be sounded out. (p. 38) 

We note that readers at a 5 th grade level or above should be able to decode words fluently 
without assistance of the kind indicated. Fifth-grade readers may have difficulty with 
vocabulary, but should not have difficulty decoding. Johnston and Young (1999) describe 
a variety of scaffolding techniques used by teachers including pre-teaching vocabulary and 
concepts and having more fluent students or themselves reading chapters aloud. 

This evidence suggests the following possibilities. In some cases, adults with low 
reading abilities, in this instance less than 5 th grade, are placed in higher level classes than 
their reading levels. More generally, we note that estimates of 25 to 40% reading 
disabilities in adults in literacy programs have been reported (Giovengo, Moore, & Young, 
1998). Learners with reading disabilities will struggle with texts regardless of their absolute 
reading levels. 

Another issue that we have begun to reflect upon is what is the appropriate 
challenge level for learners and how teachers make those judgments. In many program 
settings, adults spend relatively little educational time per week in class and therefore have 
less time overall to engage in challenging, extended, in-depth learning activities. Teachers 
may also need to be conservative in their use of challenging materials to protect, then build 
learners’ self-esteem and maintain motivation, especially in mixed ability level classes. 
Sustaining motivation in challenging materials and tasks requires more complex classroom 
management, motivational techniques, and time. We also do not know the difficulty level 
of the materials that they may be accustomed to using with adult learners in their 
classrooms. These factors may have led to a reference point for comparison different than 
the designers of this product were anticipating. These are issues of the culture of adult basic 
education and cannot be resolved solely in the design of curriculum materials. More 
research is needed to understand this issue. 

One final methodological issue we note in retrospect. Learners were only asked to 
judge “some difficulty” versus “just about right.” In the future, we may wish to ask 
whether materials were “easy,” “challenging,” or “frustrating.” This would help 
differentiate between the perceived difficulty of learners and teachers of materials relative to 
the learning goals that the materials were meant to foster. We note that there were relatively 
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higher ratings across most learners for the importance and amount learned, suggesting that 
only a small percentage may have faced frustration levels. 



Responses to the video, workbook, and online materials as stand alone 
products 

Video - The video design approach used fast-paced, documentary style footage, 
shot in actual workplaces, interviewing workers, supervisors, and human service 
personnel as the main vehicle for introducing, modeling, and maintaining the interest of the 
viewers. Each program also included staged vignettes using actors modeling problem 
solving on the job and bridging segments. A program host introduced basic concepts and 
provided bridging comments between segments. These latter two types of approach had a 
more deliberate pacing that the documentary footage. 

Johnston and Young (1999) reported that responses to the video were mixed. While 
the fast-paced, Finding a Job programs were valued for their real world, documentary 
scenarios, the Workplace Writing videos were criticized for addressing too many issues in 
a superficial manner. Johnston and Young recommend that a smaller number of topics 
could be addressed in more depth, generally sound pedagogical advice for this audience. 
The mixed responses point out the difficulty of balancing a common, thematic visual 
approach across a video series (in this case, extensive documentary footage and quick 
jumps from topic to topic) by the domain of instruction. While the techniques described 
worked in the soft skills, real world domain of the “job search skills” programs, the same 
techniques were perceived as counter intuitive to the process of becoming a good writer, 
the content of the “writing skills” programs. 

Workbooks - The workbooks followed a conventional workbook format, highly 
familiar to adult literacy teachers and learners and consistent with prior curriculum 
programs produced by KET to support their video series. The workbooks employed a high 
percentage of case scenarios sampling across work domains of service, industry, office, 
and health to help learners problem solve with authentic context problems. 

Johnston and Young reported that teachers and learners found the workbooks 
familiar, easy to teach and follow, clear in task demands, but somewhat uninteresting and 
difficult to read. We have previously discussed at length the issue of reading difficulty. The 
role of the workbooks in relation to the integrated classroom environment is still unclear. 
Are they to be used to guide pre- and post-video group discussions or as individual 
tutorials? Are they best used for introducing content, practicing skills, or both? More field 
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study of different classroom (and non-classroom) instructional environments will be 
necessary to understand better the optimal conditions for mixing and integrating all the 
media. 

Online Materials - The organization structure of the online instruction was a unit. A 
unit corresponded to each Video program/Workbook chapter, which consisted of two basic 
types of activities: video and Internet. A unit video activity began with a webpage that 
introduced a 45 sec. to 1 min. video clip from the corresponding video program. This was 
followed by another webpage of basic concepts, a webpage of activities based on the video 
clip with text boxes for saving responses to an online portfolio, and a follow-up page of 
reflection activities and suggestions for group or extended learning activities on the topic. 
The Internet activity followed the same basic structure, except that the first webpage 
introduced an external Internet site and the activities were based on tasks to do when 
visiting that website. We have already addressed the issues of redundancy in the website 
design, so they will not be repeated here. 

Johnston and Young reported that teachers and learners found the online materials 
the most appealing and exciting, and appreciated the dual benefit of learning the curriculum 
content and new computer and technology skills at the same time. A main issue for the 
future was how much prerequisite skill would benefit the learners versus learning basic 
computer motor, typing, and navigation skills as part of the curriculum. A second issue to 
address in the future is the cost versus benefit of streaming video over the web, given the 
many technical problems of speed and transfer that video caused in this pilot. 

Pedagogical assumptions underlying the writing and media design 

The approach to workplace writing in the WES Communications and Writing 
workbook describes a “process for writing” then models and scaffolds this step-by-step 
process which includes the following: (a) identify your purpose, (b) gather facts and 
organize your thoughts, (c) select appropriate format, (d) write a first draft, (e) review and 
revise, (f) proofread, and (g) distribute writing. The tasks associated with these steps were 
practiced in simulated form by answering questions, editing simulated writing, and so 
forth. Other chapters focus on types of business writing including directions, forms, 
charts, memos, and letters. The online approach to these topics employed some of the same 
principles but offered a more developmental, free writing approach to the types of writing 
activities that learners were asked to engage in. 

Johnston and Young (1999) noted a difference in pedagogical philosophy between 
the formulaic approach to writing as presented in the workbooks versus the more “process 
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writing” approach advocated in the online materials and activity. A second related issue also 
raised concerns about the limitations in the online technological support for process writing 
including limited writing space, a lack of peer review and teacher feedback, lost formatting, 
and no spelling or grammar checker. 

Concerning the difference in philosophy approach, a recent article by the new test 
specialist for GEDTS, Arthur Holbrook (1999), is noteworthy. In observing the formulaic 
similarity of the GED essays that he reads, Holbrook suggests that formulaic writing 
techniques have an appropriate place in the development of basic skills of organization and 
structure, but that overemphasizing these techniques results in writers as slaves to form 
rather than communicators. He goes on to argue that the two most important elements in 
developing a piece of writing are audience and purpose and that establishing a voice 
requires frequent writing and empowerment about one’s ability to communicate. Perhaps 
this is the pedagogical point that needs to be emphasized more concretely to teachers and 
learners. While the workbook provides some structured scaffolding as a foundation, the 
online approach has a goal of broadening the horizons at the next level of writing 
development. 

Regarding the limitations in the technology tools, two points can be made. First, as 
designers we were unsure about the value and utility that teachers and learners might fmd in 
online portfolio tools. We also had very little time to create the technology for them. 
Consequently, we provided more teacher training materials describing how and when to 
use portfolios than we did power in the actual tools. The response reported by Johnston 
and Young suggests that it will be well worth our time to invest in enhancing those tools. 

We have been investing that extra effort. We have already implemented teacher 
feedback mechanisms and are looking at online peer review tools. The issues regarding 
formatting and checking are technically more difficult for web tools, but we are 
experimenting with techniques for composing and working with word processors, then 
using the web portfolio for collecting, reflecting, and selecting. This strategy has the 
benefit of allowing learners to enhance their basic word processing skills at the same time 
asreaping the benefits of an asynchronous, omnipresent communications of the web. 
Nonetheless, we will probably have to wait for some new capabilities in the web 
technologies themselves before we can fully address users’ desires for a writing 
environment that mirrors that found in word processing applications. 

Effectiveness of the teacher’s guide 

The teacher’s guide consisted of two major parts. The first provided an overview of 
the WES multimedia program and pedagogical approaches underlying the design, with 
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special emphasis on the organization, functions, features, and navigational aspects of the 
website. The second part consisted of Before, During, and After activity suggestions for 
each of the sub-objectives and sub-objectives in each of the media (i.e., video, workbook, 
and online materials). The resulting guide provided hundreds of activity suggestions. 

Johnston and Young (1999) concluded that the teacher’s guide assembled for the 
pilot attempted to do too much and too little. It did too much in providing Before, During, 
and After activities for each and every video, workbook, and online activity in the 
curriculum. By doing so, it often provided only shorthand lesson plan notes that did not 
fully prepare teachers for the demands of classroom implementation. On the other hand, the 
introductory sections on the pedagogical approaches, how to use the website, and how to 
integrate video, Internet, and workbooks were too short. In general, the entire formative 
report raises issues of teacher preparation, technical support, and the need for additional 
assessment and management tools. All of these are important parts of the next phase of 
product development. 



ERIC 



Significance 

We knew that integrating a multimedia learning system into the adult classroom 
would require some reorganization of the classroom as well as some level of teacher 
preparation and support. The formative evaluation helped us understand how much the 
media materials needed to be revised and how better to prepare teachers to transform their 
classrooms. In another component of the LiteracyLink system, LitTeacher, courses have 
been designed to enhance teachers’ skills at integrating technology into the classroom or 
using video or Internet effectively as instructional tools. We acknowledge that we are 
creating products that challenge existing norms of teacher-led, workbook-driven 
instructional paradigms. We further admit that we are doing this in advance of wide-spread 
knowledge and skills in the field about how to implement the learner-centered, problem- 
focused, socially mediated techniques that underlay the design of the products. One benefit 
of this approach is that the materials we are creating can be used to demonstrate those 
techniques more easily than trying to adapt, for example, a traditional workbook activity. 

At the same time, we also anticipate and are supportive of teachers who apply a more 
gradual implementation strategy, perhaps integrating a single video or online activity into 
their existing curriculum, then slowly increasing the use of WES as a part of their 
curriculum. This approach was not directly tested in the formative evaluation. Regardless 
of implementation strategy, we recognize the necessity of providing extensive preparation 
and support for teachers and learners - we cannot assume that the materials themselves will 
reform or transform adult education. 
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Emerging web technologies and multimedia approaches are on the verge of 
permitting customized learning environments that are not merely automated, sequenced, 
“programmed instruction,” but rather dynamic, interactive, and collaborative settings in 
which learners can search, process, and produce knowledge. Although many in the field of 
adult education have long articulated the need to recognize adults as partners in the setting 
of goals and customizing of instruction to personal experiences, needs, and desires (e.g., 
Merriam & Caffarella, 1991), designing classroom-based instruction that embodies these 
principles has been difficult to achieve with the limited resources of most programs. 

WES and the LiteracyLink system will, for the first time, incorporate and test the 
effectiveness of many adult learning principles in a technology-based environment, and 
therefore serve as a valuable test-bed for future innovations in adult literacy instruction. 
However, the creation of customized materials raises many new issues of program and 
classroom structure and teacher training and preparation. A great deal of research needs to 
be done to understand the tradeoffs between flexibility and complexity of materials as they 
influence the design of products and the preparation for implementation in diverse settings. 
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